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ABSTRACT 

GPS-denied  indoor  mobile  mapping  has  been  an  active  area  of  research  for  many  years.  With  applications  such  as  historical  preservation, 
entertainment,  and  augmented  reality,  the  demand  for  both  fast  and  accurate  scanning  technologies  has  dramatically  increased.  In  this 
project,  we  developed  two  algorithmic  pipelines  for  GPS-denied  indoor  mobile  3D  mapping  using  an  ambulatory  backpack  system.  By 
mounting  scanning  equipment  on  a  backpack  system,  a  human  operator  can  traverse  the  interior  of  a  building  to  produce  a  high-quality  3D 
reconstruction.  In  each  of  our  presented  algorithmic  pipelines,  data  from  a  number  of  2D  laser  scanners,  a  camera,  and  an  IMU  is  fused 
together  to  track  the  3D  position  of  the  system  as  the  operator  traverses  an  unknown  environment.  This  project  presents  a  number  of  novel 
contributions  f  or  indoor  GPS-denied  2.5  and  3D  mobile  mapping  using  a  number  of  2D  laser  scanners,  a  camera,  and  an  IMU.  First,  for  3D 
mapping  we  develop  a  tightly  coupled  EKF  estimator  for  fusing  data  from  all  sensors  into  a  single  optimized  3D  trajectory.  By  formulating 
each  sensor's  contributions  independently,  we  demonstrate  a  modular  algorithm  that  easily  scales  to  an  arbitrary  number  of  2D  laser 
scanners.  In  contrast  to  existing  work  that  either  assumes  a  known  fixed  map  or  limits  the  environment  to  a  set  of  axis  aligned  planes,  we 
demonstrate  the  ability  to  map  environments  containing  horizontal  and  vertical  planes  of  arbitrary  orientation  with  no  a  priori  information. 
Additionally,  through  timing  and  complexity  analysis,  we  demonstrate  that  the  runtime  of  the  proposed  EKF  estimator  is  only  linear  in  the 
acquisition  time.  Secondly,  by  including  in  our  EKF  estimator  the  laser  scanner's  spatial  and  temporal  calibration  parameters,  we  present  a 
novel  laser  calibration  methodology.  Through  simulated  and  real-world  data,  we  validate  that  the  proposed  algorithms  are  capable  of 
calibrating  both  the  extrinsic  and  temporal  misalignments  present  in  our  system's  laser  data.  Lastly,  we  address  the  scalability  of  the 
proposed  approach  by  utilizing  a  graph  optimization  post  processing  step  that  overcomes  any  accumulated  drift  in  the  EKF  estimator.  We 
then  validate  the  proposed  3D  end-to-end  localization  system  using  3  multi-story  datasets  collected  from  real-world  environments.  The 
system's  reconstructions  are  compared  against  CAD  drawings  of  the  buildings  and  are  shown  to  achieve  an  intersection  over  union  of  over 
96%  on  all  datasets.  Lastly,  we  demonstrate  accuracy  improvements  over  our  2.5D  methods  using  a  comparison  test  against  data  collected 
with  a  static  scanner.  In  addition  to  3D  mapping,  we  also  present  a  methodology  for  2.5D  mapping  with  three  novel  contributions.  First,  we 
present  a  method  for  automatically  segmenting  barometric  pressure  data  based  on  the  floor  of  the  building  it  was  collected  from. 
Specifically,  by  using  Bayesian  non-parametrics  we  are  able  to  demonstrate  simultaneous  floor  detection  and  the  corresponding  data 
segmentation.  The  data  segmentation  is  then  used  to  extend  classical  2D  particle  filtering  across  any  number  of  discrete  building  stories. 
Secondly,  we  demonstrate  a  genetic  scan  matching  algorithm  used  to  estimate  loop  closure  constraints  even  without  an  accurate  initial 
condition.  Through  simulation  and  real-world  experiments  we  show  an  improvement  over  state  of  the  art  scan  matching  techniques.  Next, 
we  present  two  metrics  that  are  used  to  validate  the  results  of  the  genetic  scan  matching  algorithm.  We  use  both  a  correlation  and  shape 
metric  to  demonstrate  robust  and  accurate  validation  of  loop  closure  constraints  in  indoor  environments.  Lastly,  we  compare  and 
characterize  the  performance  of  the  proposed  3D  and  2.5D  mapping  techniques  developed  in  this  project.  Although  the  2.5D  mapping 
techniques  are  more  computationally  lightweight,  we  show  that  the  accuracy  of  system  is  significantly  improved  using  the  3D  mapping 
algorithm. 
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I.  Abstract 

GPS-denied  indoor  mobile  mapping  has  been  an  active  area  of  research  for  many  years.  With 
applications  such  as  historical  preservation,  entertainment,  and  augmented  reality,  the  demand 
for  both  fast  and  accurate  scanning  technologies  has  dramatically  increased.  In  this  project,  we 
developed  two  algorithmic  pipelines  for  GPS-denied  indoor  mobile  3D  mapping  using  an 
ambulatory  backpack  system.  By  mounting  scanning  equipment  on  a  backpack  system,  a  human 
operator  can  traverse  the  interior  of  a  building  to  produce  a  high-quality  3D  reconstruction.  In 
each  of  our  presented  algorithmic  pipelines,  data  from  a  number  of  2D  laser  scanners,  a  camera, 
and  an  IMU  is  fused  together  to  track  the  3D  position  of  the  system  as  the  operator  traverses  an 
unknown  environment.  This  project  presents  a  number  of  novel  contributions  f  or  indoor  GPS- 
denied  2.5  and  3D  mobile  mapping  using  a  number  of  2D  laser  scanners,  a  camera,  and  an  IMU. 
First,  for  3D  mapping  we  develop  a  tightly  coupled  EKF  estimator  for  fusing  data  from  all 
sensors  into  a  single  optimized  3D  trajectory.  By  fonnulating  each  sensor's  contributions 
independently,  we  demonstrate  a  modular  algorithm  that  easily  scales  to  an  arbitrary  number  of 
2D  laser  scanners.  In  contrast  to  existing  work  that  either  assumes  a  known  fixed  map  or  limits 
the  environment  to  a  set  of  axis  aligned  planes,  we  demonstrate  the  ability  to  map  environments 
containing  horizontal  and  vertical  planes  of  arbitrary  orientation  with  no  a  priori  information. 
Additionally,  through  timing  and  complexity  analysis,  we  demonstrate  that  the  runtime  of  the 
proposed  EKF  estimator  is  only  linear  in  the  acquisition  time.  Secondly,  by  including  in  our  EKF 
estimator  the  laser  scanner's  spatial  and  temporal  calibration  parameters,  we  present  a  novel  laser 
calibration  methodology.  Through  simulated  and  real-world  data,  we  validate  that  the  proposed 
algorithms  are  capable  of  calibrating  both  the  extrinsic  and  temporal  misalignments  present  in 
our  system’s  laser  data.  Lastly,  we  address  the  scalability  of  the  proposed  approach  by  utilizing  a 
graph  optimization  post  processing  step  that  overcomes  any  accumulated  drift  in  the  EKF 
estimator.  We  then  validate  the  proposed  3D  end-to-end  localization  system  using  3  multi-story 
datasets  collected  from  real-world  environments.  The  system’s  reconstructions  are  compared 
against  CAD  drawings  of  the  buildings  and  are  shown  to  achieve  an  intersection  over  union  of 
over  96%  on  all  datasets.  Lastly,  we  demonstrate  accuracy  improvements  over  our  2.5D  methods 
using  a  comparison  test  against  data  collected  with  a  static  scanner.  In  addition  to  3D  mapping, 
we  also  present  a  methodology  for  2.5D  mapping  with  three  novel  contributions.  First,  we 
present  a  method  for  automatically  segmenting  barometric  pressure  data  based  on  the  floor  of  the 
building  it  was  collected  from.  Specifically,  by  using  Bayesian  non-parametrics  we  are  able  to 
demonstrate  simultaneous  floor  detection  and  the  corresponding  data  segmentation.  The  data 
segmentation  is  then  used  to  extend  classical  2D  particle  filtering  across  any  number  of  discrete 
building  stories.  Secondly,  we  demonstrate  a  genetic  scan  matching  algorithm  used  to  estimate 
loop  closure  constraints  even  without  an  accurate  initial  condition.  Through  simulation  and  real- 
world  experiments  we  show  an  improvement  over  state  of  the  art  scan  matching  techniques. 
Next,  we  present  two  metrics  that  are  used  to  validate  the  results  of  the  genetic  scan  matching 
algorithm.  We  use  both  a  correlation  and  shape  metric  to  demonstrate  robust  and  accurate 
validation  of  loop  closure  constraints  in  indoor  environments.  Lastly,  we  compare  and 
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characterize  the  performance  of  the  proposed  3D  and  2.5D  mapping  techniques  developed  in  this 
project.  Although  the  2.5D  mapping  techniques  are  more  computationally  lightweight,  we  show 
that  the  accuracy  of  system  is  significantly  improved  using  the  3D  mapping  algorithm. 


II.  Peer  reviewed  Conference  Proceedings 

[1]  E.  Turner  and  A.  Zakhor,  "Automatic  Indoor  3D  Surface  Reconstruction  with 
Segmented  Building  and  Object  Elements"  International  Conference  on  3D  Vision,  Lyon, 
France,  October  2015.  [Adobe  PDF! 

[2]  R.  Zhang  and  A.  Zakhor,  "Automatic  Identification  of  Window  Regions  on  Indoor 
Point  Clouds  Using  Lasers  and  Cameras,"  submitted  to  IEEE  Winter  Conference  on 
Applications  of  Computer  Vision,  Steamboat  Springs  CO.,  March  24-26,  2014.  [Adobe  PDF! 

[3]  E.  Turner,  and  A.  Zakhor,  "Floor  plan  Generation  and  Room  Labeling  of  Indoor 
Environments  from  Laser  Range  Data,"  submitted  to  GRAPP  2014,  Lisbon,  Portugal, 
January  2014.  [Adobe  PDF! 

[4]  E.  Turner  and  A.  Zakhor,  "Watertight  Planar  Surface  Meshing  of  Indoor  Point- 
Clouds  with  Voxel  Carving,"  submitted  to  3DV  2013.  Seatle,  Washington,  June 
2013.  [Adobe  PDF! 

[5]  N.  Corso  and  A.  Zakhor,  "Loop  Closure  Transformation  Estimation  and  Verification 
Using  2D  LiDAR  Scanners, "submitted  to  special  issue  on  mobile  laser  scanning  and 
mapping,  Journal  of  Remote  Sensing,  August  2013  [Adobe  PDF! 

[6]  P.  Cheng,  M.  Anderson,  S.  He,  and  A.  Zakhor,  "Texture  Mapping  3D  Planar  Models  of 
Indoor  Environments  with  Noisy  Camera  Poses,"  SPIE  Electronic  Imaging  Conference, 
Multimedia  Content  Access.  Burlingame,  California,  February  2013.[Adobe  PDF! 

[7]  R.  Garcia  and  A.  Zakhor,  "Geometric  Calibration  for  a  Multi-Camera-Projector 
System,"  IEEE  Workshop  on  the  Applications  of  Computer  Vision  (WACV)  2013. 
Clearwater  Beach,  Florida,  January  2013.  [Adobe  PDF! 

[8]  E.  Turner  and  A.  Zakhor,  "Watertight  As-Built  Architectural  Floor  Plans  Generated 
from  Laser  Range  Data,"  3DIMPVT,  October  2012,  Zurich,  Switzerland.  [Adobe  PDF! 

[9]  E.  Turner  and  A.  Zakhor,  "Sharp  Geometry  Reconstruction  of  Building  Facades  Using 
Range  Data,"  International  Conference  on  Image  Processing,  Orlando,  Florida,  Sep. 
2012.  [Adobe  PDF! 

[10]  V.  Sanchez  and  A.  Zakhor,  "Planar  3D  Modeling  of  Building  Interiors  from  Point 
Cloud  Data,"  International  Conference  on  Image  Processing,  Orlando,  Florida,  Sep. 
2012.  [Adobe  PDF! 

[11]  S.  Lagiiela,  J.  Arnesto,  P.  Arias,  and  A.  Zakhor,  "Automatic  Procedure  for  the 
Registration  of  thermographic  Images  with  Point  Clouds,"  International  Society  for 
Photogrammetry  and  Remote  Sensing  (ISPRS),  Melbourne,  Australia,  20 12. [Adobe  PDF] 
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[12]  J.  Kua,  N.  Corso,  A.  Zakhor,  "Automatic  Loop  Closure  Detection  Using  Multiple 
Cameras  for  3D  Indoor  Localization," IS&T/SPIE  Electronic  Imaging  2012,  Burlingame, 
California,  January  22-26,  2012.  [Adobe  PDF] 

[13]  X.  Shi  and  A.  Zakhor,  "Fast  Approximation  for  Geometric  Classification  of  Lidar 
Returns,"  ICIP  2011,  Brussels,  Belgium,  September  1 1-14,  201 1.  [Adobe  PDF] 

[14]  N.  Kawai  and  A.  Zakhor,  T.  Sato,  N.  Yokoya,  "Surface  Completion  of  Shape  and 
Texture  Based  on  Energy  Minimization," ICIP  2011,  Brussels,  Belgium,  September  11-14, 
2011.  [Adobe  PDF1 


III.  Peer  reviewed  Journal  publications 

1.  R.  Garcia  and  A.  Zakhor,  "Markerless  Motion  Capture  with  Multi-view  Structured 
Light,"  submitted  to  IEEE  Transactions  on  Visualization  and  Computer  Graphics, 
January  2015,  [Adobe  PDF] 

2.  E.  Turner,  P.  Cheng,  and  A.  Zakhor,  "Fast,  Automated,  Scalable  Generation  of 
Textured  3D  Models  of  Indoor  Environments,"  IEEE  Journal  on  Selected  Topics  in 
Signal  Processing,  Volume  9,  No.  3,  pp.  409-421,  April  2015,  [Adobe  PDF] 

3.  N.  Corso  and  A.  Zakhor,  "Indoor  Localization  Algorithms  for  an  Ambulatory 
Human  Operated  3D  Mobile  Mapping  System,"  Remote  Sensing  2013,  vol.  5,  no.  12, 
pp.  6611-6646,  Oct.  2013,  [Adobe  PDF] 

4.  R.  Garcia  and  A.  Zakhor,  "Consistent  Stereo-Assisted  Absolute  Phase  Unwrapping 

Methods  for  Structured  Light  Systems,"  IEEE  Journal  on  Selected  Topics  in  Signal 
Processing,  vol.  6,  no.  5,  pp.  411-424,  Sept.  2012,  [Adobe  PDF] 

IV.  In  THE  NEWS 

October  29,  2015  -  KQED:  "Mapping  Your  World  with  a  Backpack" 

KQED  feature  on  the  3D  mapping  project 


June  8,  2015  -  Berkeley  Ensineerins:  "The  Mappins  Backpack" 

Berkeley  engineering  discuss  our  energy  audit  research. 

June  4,  2015  -  BBC  Arabic:  "Backpack  Device  Performs  Three-Dimensional  Scanning  for 

any  Buildins  Desisn  " 

Our  lab  was  visited  by  BBC  Arabic  to  look  at  our  backpack  scanning  system. 
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February  26,  2015  -  Voice  of  America:  "New  Tool  Maps  Buildings'  Energy  Efficiency" 


oice 


America  shows  off  our  latest  backpack  hardware  at  the  ARPA-E  summit. 


of 


Februan’  12,  2015  -  BERC:  "Berkeley  Based  Startups  Win  Bis  at  ARPA-E" 

Indoor  Reality  was  among  three  winners  of  a  start-up  pitch  competition  to  a  panel  of  four 
investors. 


Februan’  11  2015  -  Avideh  Zakhor  Featured  in  ARPA-E  Inspiring  Innovators  Showcase 

November  13,  2014  -  SWARM  Lab  Seminar:  "Professor  Zakhor' s  talk  on  image  based 

localization  " 

Februaiy  28,  2014  -  U.C.  Berkeley  New sC enter:  "Berkeley  Team  Takes  its  Energy  Innovation  to 

Capitol  Hill" 


A  public  affairs  story  of  our  visit  to  a  Capitol  Tech  Showcase,  held  by  ARPA-E. 

Februaiy  26,  2014  -  Energy  Manager  Today:  Backpack  Creates  Thermal  Maps 

Energy  Manager  today  ran  a  story  about  our  latest  backpack  system. 
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February  26,  2014  -  KTVU News  Sesment  on  3D  Mapping  Backpack 


The  Channel  2  News  ran  a  segment  for  our  indoor  modeling  project. 

February  26,  2014  -  EnergyWire:  "All-Seeing  Backpack  Homes  in  on  Energ\>  Waste" 


EnergyWire  article  by  David  Ferris,  E&E  reporter 

February  25,  2014  -  LBNL  Newsletter  on  RAPMOD  at  ARPA-E  Tech  Showcase 
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Lawrence  Berkeley  Lab’s  monthly  newsletter  showcased  us  presenting  our  latest  hardware  for 

our  indoor  modeling  project. 

December  10,  2013  -  FierceWirelessTech:  "UC  Berkeley  Pursues  Indoor  Positioning  Via 

Smartphone  Photos  " 

This  article  showcases  our  research  in  positioning  systems  using  just  smartphone  cameras. 

August  29,  2012  -  A  Backpack  for  BIM 

GeoDataPoint  report  by  Christine  Grahl 

V.  Presentations 

•  Keynote  speaker  for  VCIP  (visual  communication  and  image  processing)  Conference, 
November  2012,  San  Diego,  “Fast  Automated  3D  Modeling  of  Indoor  Environments” 

•  Keynote  speaker  for  ICME  (International  Conference  on  Multimedia  Exposition),  July, 
2011,  Barcelona,  Spain 

•  Invited  presentation  at  SPIE  Electronic  Imaging  Conference  on  Image  Based 
Localization,  February  2016,  San  Francisco,  CA 

•  Invited  talk  at  SPAR  International,  Coloroda  Springs  on  3D  modeling  of  indoor 
environments,  April  16th  ,  2014.  •  Invited  talk  at  Center  for  Built  Environment,  Berkeley 
CA  on  3D  modeling  of  indoor  environments  with  applications  to  building  energy 
efficiency,  April  24,  2014. 


VI.  Honors  and  Awards 
Best  paper  award  for  the  paper 

R.  Garcia  and  A.  Zakhor,  “Markerless  motion  capture  with  multi-view  structured  light”,  SPIE 
Electronic  imaging  conference  on  3D  Image  Processing,  Februay  2016,  San  Francisco,  CA 


VII.  Patents  Awarded 


VIII.  Personnel  Supported 

Avideh  Zakhor,  Professor 
Nicholas  Corso,  Graduate  student 
Eric  Turner,  Graduate  student 
John  Kua,  Staff  member 
Richard  Zhang,  Graduate  Student 
Victor  Sanchez,  Postdoc 
Ricardo  Garcia,  Graduate  Student 
Peter  Cheng,  Graduate  Student 
Plamen  Levchev,  Staff  Member 
N.  Kawai,  Postdoc 
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IX.  Graduating  Undergraduate  Metrics 

•  Eric  Liang  currently  graduate  student  in  computer  science  at  CMU 

•  Jason  Liang  currently  graduate  student  in  ECE  at  University  of  Texas,  Austin 

•  Gurshamnjot  Singh  currently  working  at  Intel 

•  David  Zhang,  currently  working  at  Apple 

•  Vaishaal  Shenkar,  currently  graduate  student  in  computer  science  at  UC  Berkeley 

•  Mark  Jouppi  currently  working  at  Goolge 

•  Chaoran  Yu,  currently  working  at  Bloomberg 

•  Andrew  Zhai  currently  working  at  Pinterest 

•  Eric  Tzeng,  currently  graduate  student  at  UC  Berkeley 

•  Christopher  Dinh,  graduating  May  2017  from  UC  Berkeley 

•  Raphael  Townshend,  currently  a  graduate  student  at  Stanford 


X.  Master’s  Degrees  Awarded 

1  E.  Turner,  "Watertight  Floor  Plans  Generated  from  Laser  Range 
Data,"  Master's  Project,  Department  of  Electrical  Engineering  and  Computer 
Sciences,  University  of  California,  Berkeley,  May  2013.  [Adobe  PDF! 

2  N.  Corso,  "Loop  Closure  Transformation  Estimation  and  Verification  Using  2D 
LiDAR  Scanners,"  Master’s  Project,  Department  of  Electrical  Engineering  and 
Computer  Sciences,  University  of  California,  Berkeley,  May  2013.  [Adobe  PDF! 

3  P.  Cheng,  "Texture  Mapping  3D  Models  of  Indoor  Enviornments  with  Noisy  Camera 
Poses,"  Master's  Thesis,  Department  of  Electrical  Engineering  and  Computer  Sciences, 
University  of  California,  Berkeley,  December  2013. 


XI.  Scientific  Progress  and  Accomplishments 

1.1  Automatic  Indoor  3D  Surface  Reconstruction  with  Segmented  Building  and  Object 

Elements 

Automatic  generation  of  3D  indoor  building  models  is  important  for  applications  in  augmented 
and  virtual  reality,  indoor  navigation,  and  building  simulation  software.  This  paper  presents  a 
method  to  generate  high-detail  watertight  models  from  laser  range  data  taken  by  an  ambulatory 
scanning  device.  Our  approach  can  be  used  to  segment  the  permanent  structure  of  the  building 
from  the  objects  within  the  building.  We  use  distinct  techniques  to  mesh  the  building  structure 
and  the  objects  to  efficiently  represent  large  planar  surfaces,  such  as  walls  and  floors,  while  still 
preserving  the  fine  detail  of  segmented  objects,  such  as  furniture  or  light  fixtures.  Our  approach 
is  scalable  enough  to  be  applied  on  large  models  composed  of  several  dozen  rooms,  spanning 
over  14,000  square  feet.  We  experimentally  verify  this  method  on  several  datasets  from  diverse 
building  environments. 
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Figure  1:  An  area  modeled  by  our  technique:  (a)  a  photo  of  the  room;  (b)  the  volumetric  boundary  of  room;  (c)  final  mesh 
with  room  and  objects  modeled;  (d)  final  mesh  of  room  only,  colored  by  planar  region. 


Figure  2:  The  scanned  volume  is  meshed  using  two  ap¬ 
proaches  that  are  combined  to  separate  room  geometuy  and 
object  geometry,  The  complex  geometiy  from  the  octree 
(upper  left  in  ted)  and  the  simple  geometiy  from  the  2.5 D 
model  (lower  left,  in  blue)  are  combined  to  extract  the  ob¬ 
ject  volume  (upper center,  in  green)  and  the  building  volume 
(lower  center,  in  grey),  These  volumes  are  meshed  sepa¬ 
rately  and  exported  (right,  in  black). 


Figure  7:  Example  meshing  output  of  residential  area:  (a)  photo  of  area;  (b)  all  reconstructed  geometry;  (c)  geometry  of 
room  surfaces  only,  colored  by  planar  region. 
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(a)  (b) 


Figure  8:  Example  scan  of  equipment  in  hospital’s  hybrid 
operating  room:  (a)  picture  of  scanned  area:  (b)  object 
model  triangulation  of  operating  table  and  equipment 


1.2  Loop  closure  transformation  estimation  and  validation 

In  many  simultaneous  localization  and  mapping  (SLAM)  systems,  it  is  desirable  to  exploit  the 
fact  that  the  system  is  traversing  though  a  previously  visited  environment.  Once  these  locations, 
commonly  known  as  loop  closures,  have  been  detected  the  system  must  be  able  to  both  compute 
and  verify  the  relative  transformation  between  proposed  locations.  In  this  paper  we  present  two 
independent  algorithms,  using  2D  LiDAR  scanners,  for  robustly  computing  the  transformation 
between  arbitrary  locations  with  overlapping  geometry  and  validating  the  resulting  transforms. 
First,  a  scan  matching  algorithm  based  on  a  genetic  search  and  a  fractional  distance  metric  is 
presented.  Secondly,  two  metrics  are  proposed  to  verify  the  recovered  transforms.  Through 
experimental  results  the  proposed  algorithms  are  shown  to  robustly  estimate  and  validate  loop 
closure  transformations  for  both  manually  and  automatically  defined  candidates. 


11 


Final  Report  for  ARO  Sponsored  Project  on  3D  Indoor  Modeling  for  5/1/2011  to 
6/30/2015;  P.I.  Avideh  Zakhor,  U.C.  Berkeley 


Figure  17.  Results  of  applying  the  end-to-end  system.  (a),(d),(e)  The  occupancy  grid 
maps  that  result  from  the  RBPF  algorithm.  (b),(e),(h)  The  dead  reckoning  trajectories  with 
validated  loop  closure  constraints  overlain.  (c),(f),(i)  The  3D  paths  viewed  from  the  top 
down  after  optimization  has  been  applied. 
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1.3  Simplified  Floor  Plan  Modeling  and  Room  Labeling  [3] 

Automatic  generation  of  building  floor  plans  is  useful  in  many  emerging  applications,  including 
indoor  navigation,  augmented  and  virtual  reality,  as  well  as  building  energy  simulation  software. 
These  applications  require  watertight  models  with  limited  complexity.  In  this  paper,  we  present 
an  approach  that  produces  2.5D  extruded  watertight  models  of  building  interiors  from  either  2D 
particle  filter  grid  maps  or  full  3D  point-clouds  captured  by  mobile  mapping  systems.  Our 
approach  is  to  triangulate  a  2D  sampling  of  wall  positions  and  separate  these  triangles  into 
interior  and  exterior  sets.  We  partition  the  interior  volume  of  the  building  model  by  rooms,  then 
simplify  the  model  to  reduce  noise.  Such  labels  are  useful  for  building  energy  simulations 
involving  thermal  models,  as  well  as  for  ensuring  geometric  accuracy  of  the  resulting  3D  model. 
We  experimentally  verify  the  perfonnance  of  our  proposed  approach  on  a  wide  variety  of 
buildings.  Our  approach  is  efficient  enough  to  be  used  in  real-time  in  conjunction  with 
Simultaneous  Localization  and  Mapping  (SLAM)  applications.  Examples  of  this  approach  are 
shown  in  the  figures  below. 
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2D  Wall  Samples  Generated  Floor  Plan  3D  Extrusion 


(a)  12,823  Wall  Samples  6,084  Triangles 


(c)  12,183  Wall  Samples  4,912  Triangles 


3  Rooms  7.0  Sec 


(d)  2,957  Wall  Samples  884  Triangles  5  Rooms  1.2  Sec 


(e)  12,125  Wall  Samples  3,604  Triangles  7  Rooms  4.5  Sec 


Figure  2:  Floor  plan  construction  from  estimated  position  of  wall  samples.  Left  column 
indicates  wall  samples,  middle  column  indicates  generated  2D  floor  plan,  and  right  column 
indicates  3D  extrusion  of  building  model  from  floor  plan,  (a)  Mid-sized  cubicle  area,  (b)  hotel 
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lobby  and  hallways,  (c)  student  offices  and  cubicles,  (d)  small  office  complex,  (d)  academic 
building  hallways  and  cubicles. 

1.4  Floor  Plan  Generation  from  3D  point  clouds  [8] : 

We  have  developed  an  algorithm  that  generates  as-built  architectural  floor  plans  by  separating 
the  floors  of  the  Li-DAR  scan  of  a  building,  selecting  a  representative  sampling  of  wall  scans  for 
each  floor,  and  triangulating  these  samplings  to  develop  a  watertight  representation  of  the  walls 
for  each  of  the  scanned  areas.  Curves  and  straight  line  segments  are  fit  to  these  walls,  in  order  to 
mitigate  any  registration  errors  from  the  original  scans.  This  method  is  not  dependent  on  the 
scanning  system  and  can  successfully  process  noisy  scans  with  non-zero  registration  error.  Most 
of  the  processing  is  performed  after  a  dramatic  dimensionality  reduction,  yielding  a  scalable 
approach.  We  demonstrate  the  effectiveness  of  our  approach  on  a  three  story  point  cloud  from  a 
commercial  building  as  well  as  on  the  lobby  and  hallways  of  a  hotel.  An  example  of  the 
application  of  this  method  is  shown  in  the  figure  below: 
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Figure  9.  (a)  Full  point  cloud  for  three-story  model,  taken  with  mobile  scanning  system;  (b-d)  Processing  of  each  stoty,  with  (left  to  right) 
wall  sample  locations,  triangulation  labeling,  watertight  curve-ht  model,  and  comparison  against  ground-truth  blueprints. 


1.3  Watertight  Planar  Surface  Reconstruction  with  Voxel  Carving  [4] 


3D  modeling  of  building  architecture  from  point-cloud  scans  is  a  rapidly  advancing  field.  These 
models  are  used  in  augmented  reality,  navigation,  and  energy  simulation  applications.  State-of- 
the-art  scanning  produces  accurate  pointclouds  of  building  interiors  containing  hundreds  of 
millions  of  points.  Current  surface  reconstruction  techniques  either  do  not  preserve  sharp 
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features  common  in  a  man-made  structures,  do  not  guarantee  watertightness,  or  are  not 
constructed  in  a  scalable  manner.  We  have  developed  an  approach  that  generates  watertight 
triangulated  surfaces  from  input  point-clouds,  preserving  the  sharp  features  common  in 
buildings.  The  input  point-cloud  is  converted  into  a  voxelized  representation,  utilizing  a 
memory-efficient  data  structure.  The  triangulation  is  produced  by  analyzing  planar  regions 
within  the  model.  These  regions  are  represented  with  an  efficient  number  of  elements,  while  still 
preserving  triangle  quality.  This  approach  can  be  applied  to  data  of  arbitrary  size  to  result  in 
detailed  models.  We  apply  this  technique  to  several  data  sets  of  building  interiors  and  analyze  the 
accuracy  of  the  resulting  surfaces  with  respect  to  the  input  point-clouds.  An  example  of  this 
method  is  shown  in  the  figure  below: 


Figure  2:  Top-down  view  of  surface  reconstruction  of  a  warehouse-sized  retail  shopping  center. 
Each  planar  region  given  a  random  color.  Generated  with  resolution  of  10  cm. 
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Figure  3:  On  left,  surface  reconstruction  of  a  10.5  m  x  9.5  m  conference  room  with  table  and 
chairs,  at  resolution  of  5  cm.  On  right,  the  input  point  cloud. 


Figure  4  :  Input  point  cloud  of  construction  area,  colored  by  depth  from  camera 


Figure  5:  Generated  surface  reconstruction,  showing  triangle  elements.  Resolution:  5  cm. 
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